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ABSTRACT 


An approach to the classification problem is one that is dependent 
On the amount of information assumed to be known about the distributions 
of the populations. It is assumed in this thesis that nothing is known 
about the distributions for a two population case. The probability of 
misclassification of an individual Z is presented in general. The 
approach is carried further to develop explicit forms of the error 


probability when the two populations are bivariate exponential. 
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1. INTRODUCTION 

The discrimination problem may in a sense be considered 
that of multiple classification, i.e., an individual Z is 
known to belong to just one of j categories or populations 
and it must be classified into one of these populations on 
the basis of what is known about Z and the existing populations. 
The problem lends itself to a statistical approach when avail- 
able information about Z is in the form of observed values of 
random variables which have probability distributions for each 
of the different categories or populations. 

Discriminatory analysis in its historical evolvement is 
documented with an extensive bibliography by J. L. Hodges in 
(1). 

The history of discriminatory analysis may be represented 
in several broad phases of development. A Pearsonian stage is 
identified with the introduction and use of his coefficient of 
racial likeness. This stage is considered to be followed by 
a Fisherian stage associated with the introduction of the linear 
discriminant function and this stage is followed by a Neymarm 
Pearson stage and a contemporary Waldian stage. The latter two 
reflect the introduction of the concepts of probability of 
misclassification and that of risk into the realm of discrim 
inatory analysis. 

For dep Lehey and ease of computation the discrimination 


problem will be considered only in the two population case, i.e., 


the individual Z is known to be distributed over some space 
according to distribution F or according to distribution G and 
it is desired to decide which of the distributions Z has on the 
basis of the observed value z. 

An approach to this classification problem is one that is 
dependent on the amount of information assumed to be known about 
F and G. This approach allows the problem to be segmented into 
three types: 

(1) F and G are completely known -- On the basis of an 
observation of Z, the problem is to determine which is the dis- 
tribution of Z. Treat of this problem has been extensive 
and its solution lies within the Neyman-Pearson lemma. 

(2) F and G are known but complete knowledge is lackim 
in its parameters -- F and G are of the same family of dis- 
tributions but differ parametrically and on the basis of an 
observation on Z, the problem is to determine which is the dis- 
tribution of Z. Hodges and Fix discuss this in [2] and identify 
the most familiar example of this process as the linear discrim- 
inant function where the assumption is made that F and G are 
p-variate normal distributions having the same (unknown) co- 
variance matrix. It is noted that the approach is reasonable 
if the assumptions are well founded but validity is questionable 
if the populations are obviously not normal or if they are normal 
but with obviously unequal covariance matrices. 


(3) EF and G are completely unknown -- Nothing is assumed 


about F and G other than their existence and on the basis of an 
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observation on Z, the problem is to decide which of F and G is 
the distribution of Z. 

The last problem type is a problem of nonparametric class- 
ification and is the area of concern of this thesis. The area 
of nonparametric classification has its possibly first published 
treatment in Hodges and Fix's [2] and [3]. In these papers 
Hodges and Fix considered the two population problem, however, 
they noted that the approach ig general, has optimum properties 
for large samples and applies to cases where there are more 
than two populations to be discriminated. In [3] a comparison 
is made of the nonparametric approach and the linear discrin- 
inant approach, assuming both populations to be normal with 
equal covariance matrices. 

Eaton in [4] and Hager in [5] extend the work of Hodges 
and Fix to one dimensional exponential populations. This paper 
continues the investigation of exponential populations when the 
distributions are bivariate exponential. 

Section 2 will introduce and summarize the concepts and 
methodology of [3]. Section 3 will apply these concepts to 
calculate a probability of misclassification utilizing two 
different distance functions to the two population problem when 
both populations are bivariate exponential. Section 4 will 
present the conclusions and recommendations evolving from the 


effort set forth in Section 3. 
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2. PERFORMANCE OF A NONPARAMETRIC DISCRIMINATOR WHEN THE TWO 
POPULATIONS HAVE NORMAL DISTRIBUTIONS WITH 
EQUAL COVARIANCE MATRICES 


The two population classification problem will be defined 


in the following manner. Let Xi» ae ee*, X be a sample from 
n 


a p-variate distribution F and let Y., Yo» ia a be a sample 


1 
from the p-variate distribution G. An observation z is made 

and it is known to be distributed either as F or as G. The 
problem being to assign z to one of these two. 

The approach utilized for the nonparametric proé@edure will 
be through the concept of nearness. A distance function is 
defined in the p-dimensional sample space which allows a ranking 
of the combined samples according to their nearness to z. With- 
in this framework, z would be assigned to the F population if 
most of the nearby observations are X's and assigned to G if 
most of the nearby observations are Y's. For simplification, 
the sample sizes are assumed to be equal, i.e., m=n. As an 
assignment criteria, an odd integer k is selected and z is 
assigned to that distribution from which came the majority of 
the k nearest observations. The case studied in this thesis 
will be when k = 1, the rule of the nearest neighbor. 

In [2] it is shown that several of these nonparametric 
discriminators have asymptotically optimum performance as m 
and n tend to infinity. By this is meant that probabilities 


of misclassification, 
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A = P (Z is assigned to G | Z came from F) 


2 = P (Z is assigned to F | Z came from G) 


will approach the theoretical minimum values obtainable if F 
and G were completely known as the sample size (m,n) tends 
to infinity. 

In [3] Hodges and Fix investigate the probabilities of 
misclassification of the nonparametric procedure when the 
sample size is small and compare the results with the linear 
discriminant function probabilities of misclassification. The 
populations are assumed to be normal with equal covariance so 
the linear discriminant function is optimal and the nonpara- 
metric procedure can be compared against the optimum for a 
comparison as to how much discriminating power is lost when 
the sample sizes are small. 

This Section will present some of the concepts and results 
of [3] developed in establishing this comparison. 

Initially, it is stated that the problem can be reduced 
by considering linear transformacions on the observation space 
so that F and G will always have the identity covariance matrix, 
iee., the p transformed measurements are independent in each 
population and that each measurement has unit variance. Also 
the expectation vector of F can be placed at the origin and 
that of G on the positive first axis. Thus only two parameters 
p and dA are required to specify the transformed populations 


where 
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X = —£ (first coordinate of Y) 
= distance between the means Of the transformed 
populations. 

For the linear discriminant function, BP, and P, are 
unchanged by this transformation, hence there is no loss of 
generality. a and 7 for the nonparametric discriminators 
are also unaffected since such linear transformations map the 
totality of possible distance functions of the original space 
One to one into the totality of the new space. 

The univariate normal case is considered in depth due to 
its computational simplicity both for the linear discriminant 
function and the nonparametric procedure. 

Considering the linear discriminant function first, the 
univariate case eliminates matrix computation and allows the 
classification problem to be stated as; compute the arithmetic 
mean ot the sample means, ae » and assign Z to the population 


whose sample mean lies on the side of (Xx + Y)/2 as does z 


itself. An error is committed then if and only if; 


< 
V 
~ 


Z>(X+yY)/2 andy 
or 
Z<(X+yY)/2 and Y < x. 


Hence, 


Bae P(Z > (X+¥)/2, Y > X) + P(Z < (K+Y)/2, Y < X) 


and Py pul PS due to the symetry of this particular problem. 


In [3] it is shown by defining new variables of X, Y and Z, the 
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limiting value of P, as n> is .5. 


1 
Considering now the nonparametric procedure, again n=m 

and the populations are univariate normal. The discriminator 

will be the case of k= 1, i.e., assign Z to the nearest sample 

(rule of the nearest neighbor}. The distance function utilized 

to measure the degree of nearness will be; 


P 
Q (x,z) = max Ix, - Z 


i=] 1 


This function, A (x,z}, describes a hyper-cube in p space and 
is only one of many functions that could be used. In the case 
of p = 1, A (x,z)} corresponds to Euclidean distance. 

To arrive at the error probabilities a conditional prob- 
ability PL is introduced and defined as the probability that 
the nearest of the 2n sample observations to Z is a Y given that 


Z= 2. Then 


(1) ~~ ge (2)] = [ £22) P (2) dz 


a 
where the distribution of Z, 


(2) = QV 27) exp (- > Zoi 


is the distribution of the X's, F. 
q 
In order to calculate P (2) in general the quantities H,(6 ) 


and K (6) are defined as follows: 
Z 
4 
Ss ) = P (the distance of X from z < 6 ¥ 


Ho (8 ) = P (the distance of Y from z <6 ). 
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The explicit forms of Hg ) and Ke) and therefore Py will 


be dependent on the distance function utilized to express the 
nearness. 


In [3] the distance function utilized is 
P ‘ 
A (x,z) = max EA - z, | 
i=l 
and since the univariate case is considered, H, (6 ) and K (8 ) 


can be expressed as: 


i 
H(6) = P ([x-2| <6 ) 


HH 


K (8 ) P (|y-z| <6) 

The formulation of Be in terms of HC ) and K ) can be 
done by considering the distance from z to each of the Y obser- 
vations, 


y-els Ie-el sors Weyeel 


Hence with the assumption of independence and of equiprobable 


events, 
rk | 
P(max ly, -2 < é ) = P(|y,-z| <8) “a's 


P(Y-z| <4) =(<,( @)*. 


4 


Therefore, 


%, 


P(min l¥,-2 > Fy) - CIEE ~z| > 5 yore 


e(ly -2| >6) = (-K.@))" 
n A 


or 


p(min |¥,-2| <6) = 1 -(-K,(6))". 
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The density of the minimum distance between Y and z is then 

n (KG)? ake . 

(6 )) AC? 
Similarly the minimum distance between X; and z can be treated 
with the result that 
ft n 
P(min IX -2| > 6) = (1-H (8)) . 
Now, P (2 = P (nearest observation to Z is a Y given that 


= : # 
ZZ) = | P(min Ix, -2 > 6 | min ly, -2| = 6 Up 
° 


minly, -z| 6°) ag. 


ot 
In terms of Hy (' ) and K, t ) then, 


i 


1 . 2 ak Gy «dg 
(2) Pa) =n] CHAE)" KCB) aK (8) 
0) 


Equations 1 and 2 form the basis for computations for the 
tule of nearest neighbor for any p and any distance function, 
3 explicit form of Hy (18 ) and KGS ) being dependent on the 
distance function. 

Figures 1 and 2 illustrate the comparison of the linear 
discriminant function and the nonparametric discriminator error 
probabilities when the distance function is A, k = 1 and the 
populations are univariate normal. 

It is shown in [3] that for large n and in general, 


pte | 2(z) \- i g(z) £(z) dz 
1 £(z)+g(z) o  £(z)+g(z) 


the value which by Schwartz's inequality cannot exceed 4 . 
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A=l1 
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eee Kz 2 
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So ee Mee el ee ee! ce eee Oe We 
aly 2 3 5 ro 2om0. 50 Loe 
n 
FIGURE 1 


Probability of Error Pj vs. n for Linear Discriminant 


Function and Nonparametric Discriminator, k= 1. 
Both populations are univariate normal. A’ = distance 
between the means. Distance function is A. 
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e) il 2 5 4 


FIGURE 2 
Probability of Error P. vs. A for Linear Discriminang 
Function and Nonparametric Discriminator, k = l. 
Both populations are univariate normal. Dotted line 


indicates nonparametric vroceduree n=tI1 is identical 
for bothe Drstance uneven ig AS 
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The dependence of Po on the distance function was mentioned 
earlier and Hodges and Fix in [3] present some results showing 
how P, is affected by alternative distance functions. 

In considering these other distance functions, the sample 
populations are assumed to be bivariate normal allowing a greater 


choice of distance functions than can be encountered when p=1l. 


The function A (x,z)is in this case, 


D(x, 5%) + (2 ,929)) = max (1x,-2] 5 [x,-2,1) 


a locus of points centered at z in the form of a square. 
Euclidean distance, describing a circle centered at z is 


defined as, 


2 2. % 
OCG. 4x z5z)) = ((X¥.-Z2) + (% -Z) ) ; 
jj SS pe Sho. ry i 
A distance function describing a rectangle centered at z 
in the ratio of one to three and who’e sides are parallel to 


the axes is, 


O, C(x, 08)» (2, .29)) = max (1X,-2,] , 31X,-2,1) 


1°72 
i.,e., a rectangle whose vertical dimension is three times its 


horizontal dimension and the common multiple being the 


max (1x, -2, : Ix, -2, | )- 


Similarly a distance function describing a rectangle 
centered at z in the ratio of three to one and having sides 


parallel to the axes is, 


4, ((%) 5X5) > (z,»2,)) = max (31x, -2, | ’ Ix, -z, | ) - 
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Limited computed results are offered in [3] regarding the 
use of thege distance functions A, A. aD and a - The com 
parative results are illustrated in Figure 3 in the form of a 
Biot P (0,0) and n. P (0,0) is the conditional probability of 
Error given that z is at the origin. It was presented because 
it was remarkably consistent with the value of ‘- 

Comparison of the results in Figure 3 concludes that in 
this case there is little difference in Ee whether A or - is 
used. However there is great effect with respect to the use 
of the other distance functions and hence a burden is placed 
upon the statistician for selecting the appropriate distance 
function. 

Though not covered in this summary, Hodges and Fix in [3] 
investigate to a limited extent: 

(1) The nonparametric discriminator using A as a distance 
function with k = 3 for the univariate and bivariate normal 
distributions. 


(2) The nonparametric discriminator using A as a distance 


function with k = 1, n= 1, and p 2 2. 


rad 


P, (0,0) 
se) 


a2 


ec 





aL 2 3 4. 


n 
FIGURE 3 
Probability of Error P_(0,0) vs. n for Various Distance 


Functions for the Nonparametric Procedure, k = l. 
Botn populations are bivariate normal, A = 2, 


ZZ 


3. NONPARAMETRIC CLASSIFICATION ERROR PROBABILITIES FOR TWO 
DIMENSIONAL EXPONENTIAL POPULATIONS FOR 
TWO DISTANCE FUNCTIONS 

Hodges and Fix's work in [2] and [3] of comparing the 
performance of the linear discriminant function and that of the 
nonparametric procedure is applied to the two population expo- 
nentially distributed case by Eaton in [4] and Hager in iS }e 
Eaton studies the small sample performance and Hager extends 
Eaton's work to larger sample size and also gets some asymp- 
totic results. In both [4] and [5] the study is limited to one 
dimension, i.e., p = 1. Hodges and Fix's major effort was for 
the case p = l, with only limited presentation of results for . 
p = 2 for the nonparametric procedure and none for the linear 
discriminant function. 

This section will consider the following: 

(1) The two population classification problem when the 
distributions are two dimensional exponential. 

(2) Formulation of the error probability Ps = P(Z is 
assigned to G\z came from F), utilizing the nonparametric pro- 
cedure when the distance function is: 


(a) Hyper-cube, 


P 
A (x,z) = max [|x -z.]] 
i=l 1 + 


(b) Hyper-sphere, 


p ee 2 ,% 
A, (x25) rae A Ss ae J 
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The density functions of F and G will be 


x 


1 Xo 


(x, »x ) 
2 


respectively where 


£ 


X X 


and 


Independence is assumed between 7 and x 


Y. 
2 


Following the procedures of Section 2, 


1 o 


# = J J P {62 a ASE »Z 2 dz, dz, “ J P 62) f, Baz. (z, »z ,) dzod 


Ze 
2 


* 22 Zo 


where 


f Gz = tae é 


Z,45 


io) 


p {c “E 
6) > 5 Ls H, (6) J 1 - K, (6) J 


0 


1 2 


1 2 


wil 


Yy (y, ’ se 


= Q 


and i 


r 


Lb 


0 


Yi" 0 


io @ 


e 
12 


“0 
(2, 
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y 6 »¥) 
¥ 


-(A +A 
( 1 2*,) 


-Gy, “5 by, 


ce c 
Pp = P(z) £ (z.,z2 ) dz dz 
J J 1 Z%o i 2 1 2 


+r 
272 


n-1 


otherwise 


otherwise. 


and also for = and 


1 


for z,A. 20 
L 1 


otherwise 


dk, (8) 


denoted by 


FO%e X Bale 
7 


for y ,» 20 
1 i 


Be Ete (al ; 


oy 


s n n-l 
ae =n (1-0, (5) [ 1-K,(5) J ae 


&S<ez< 
hee 


¢ n| (1-H, (6)}°L1-K, (6) ]" a) 


<§ @ 
zy § ™ 


n n-l 
:7 (1-H,(6)] C1-K,(8)] ak, (6) 


—o « 
“1% 


when Z 2 ei: When aa Zo the subscripts interchange. 


The explicit form of HO) and Bet) will be dependent 
on the distance function utilized and the relationship of & 
and Zz. 

The hyper-cube distance function will be considered first, 


2 
A (ae Tes C|x.-z.]] A A (y.z) = max Cly.-z,|) ° 
i=] 1 1 izl 1 


For Z=z, and 6 2 0, H (6) and K, (6) can be defined. 
Zz 


H (6) =1P¢ mee Ci hne-z. | J Oh a~ Bemaxl lez), lx -2 |] <6) 
Z i=] r 1 : & a 2 


+ 6 


- P(Ix,-2, 1, IX,-29 Ses P(1x,-2. | a P(Ix,-2,| ee. 


assuming independence of the differences. Hence H, (8) will be 


evaluated in six regions: 
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z.+ § z+ 6 


1 2 
ani -e- 
HAC) =) fy GD ox, i fx (H) dx, 6 S205 2, 
z- § z.- § 
1 2 
+ § 
* z+ § 
uf < 6s 
a | 7 aay dx, } ‘2 dx, 2. “ 2 > 5 
z.- 6 0 
1 
z+ 8 z+ 6 
1 2 
: I 
= xs s s 
J f. (x, ae J fy (x,) dx, zszs 6 
1 o | «(6@ 
| 
+ § + § 2 
Zz z, 
< 
2 | a (x,) dx, | f. (x,) dx, Oa Lee 
1 2 
Fn as z- 6 
1 
; - : 
me 2 
= £ | £ (x) dx 2s 6s 2 - = 
Jf, &) éx, Mee D1 25 ee 
1 2 | 
re) z- 6 t 
2 
+ 6 
Z Be § 
f x dx Zs s 6 :¥ 


Oo Oo | ? 
Similarly 


K (8) ~ P@Rgez <6) P(I¥,-z.| <6) 


and will be defined as was H, (5) in the six regions. Differ- 


entiating ©) yields a, So) in the six regions. 
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Evaluating HR) K_ (4) and aK, (5) and combining terms 


yields, when zy 2 

Zz 

2 -A_z,-A,z n 

PC) ay a { ing 6 1 22 sinh ij 6 sinh r 6) 

a 2 

+). 2. =,2 
2 n-l 
[1-4e +1 2 ote ¥ 5 sinh by 6 | 


“b,Z,-b_Z 
1 
(4e : 7?) (u, sinh }, Snes w, 6 


+t: “e by 6 cosh M, 6 ] dé 


vA % 
-hyZy-Aoz -v\. 6 


1 
-\1zZ n: 
+n{ (1-2. et es kh, 6+2e 22 sinh r,6] 
Zz 


2 


“H4z “Hb ZZ, - 5 iv 1 
[1-2 e . sinh 4,6 +2e pio 222 sinh aS 


“24 “Hy 24 - UgZ-Hoo 
[ 2u,e cosh ahs - de i oe 


We oP 
cosh 4,6 + 2,e 22 2 sinh; 145 } dé 


Le) 
nr 27h? A ae 


"ea fl 


o) 


“46 -i -\,6-A.6 n 
1 1 
+ e 23 


“Hy Zp yS Hoy HZ wz - yd - Lo n-1 
P — 20270, 11 oa = 24 


“42, -b,5 “pp zZ,-y_6 “wb 2c 2 bb, b- 5 
Lue a aw 22 2 -(u,+ ua)e b1'22 1 2 446 


2/7 


and when z 2 z 


2 


z 
i -i 
a A = a | [1-4 e 


Oo 


Gore ey Ye 


“By 2) ~z 
[4 e s “(, 


Z 
2 ae 
| ae 


a 


~U 9% 


(1-2 e Sine EA° 


ad © 
a2 cosh wp 6 


[ 2uoe F 


osh p 6 
c M, 


[eo] 
“Az 1-Ay6 
a Ce - + e 
a) 


Z 


= “1.6 ~ 
eet 2 2 


46 


ey 
[ a a 
we 


1217524 


sinh ccs LL 


202 sinh A8 + 20 


“hazy 


56 = 
2 


Thg5 ~U 6) 


n 
sinh } 6 sinh A_6] 
2 1 


n-1l 


22 sinh oe sinh wu 15 


i° + Lei 46 


cosh 5) }d6 


SN sods OTN ‘ 
er 2%2" 1 sinh 6] 


n-l 


“PZ yb “46 
<- sinh 16] 


+ 2e 


/ 


“lb. 247k Z 
11 9 


-1,6 
- Qe 21 


ree 1 Wee, Hae 


+ 2 e sinh » 6] dé 


-A 5-4 § = 


“Ay 2yrAgZo-AyO-4, 


2H 


-y z,-.6-p 6 n-l 
1 yoo wl a iy 


“by By Hg Zo yO “U6 


2 
5 (u nS) e€ 


146. 
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d 


Then P = [ ne 
1 J if J P (z) Aye dagea, + 


z 
p 22] 0 “A, z 
De }  P (z)rV,e : dz_dz 
1 1 
° ° 


The Euclidean distance or hyper-sphere distance function 


for p = 2 is: 
2 2 2 2 
4 (xyz) = L (x, -z,) + Come } : A (yz) = L&, -24) + (X5-Z5) }" : 


Evaluation of a P (2), H, (8) , K, (8) and dk, (6) will follow 


the same logic as that followed for the hyper-cube. The explicit 
form will be more awkward however. 


Consider HL () = P([ (K,-24)° + (x,-2,)71" <§&). For ease 


of notation define: 


i 
a 
’ 
N 


S =X -2 andT 
l 1 


Then, 9 2 , 
H (6) = P(S?7 +7 <6) =| | fereeey Ge) ae 
Z ST 
g2+r2 < 67 
where 
“Ay (stz)) 
f (s) = f, (stz_) = AVe fors+z 20 
S X, 1 A 1 
= 0 otherwise 
“Aj (ttz,) for t + - 20 
f = +p = \ 
7? tt z5) e 
= 0 Otherwise. 
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The definition of § and T translates the axes to center at 

(2 53,) and Be) is the evaluation of the circle centered at 
ee? with 6 2Q. Here as before, evaluation of H (5) will 
be over a region which is the union of six mutually exclusive 


sub-regions. Therefore, 


6 (s?_,2 
H (6) = 4f f szs 
2S ) i} gr 682 t) ds dt § ZS z, 
oO 
§ 62-54 
= <6 
| | fon (sot) ds dt zs 8 <2, 72 
-§ "Zo 
§ 52.7 
= s,t}) ds dt zszs § 
| J fon rt) 2 1 
"24 “23 
6 leeoee 
= | [ f (s,t) ds dt 6s zis z 
ST 2 
o Oo 
8 67-57 7 | 
~ ‘= 
= i fon(s,t) ds dt 2cMe< ge 2 
J a 1 ye 22 “2 
Relies | 
6 (52_,2 
< a4 < 6 
i { £ op 65>) ds dt z, = 
tH, > 
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K Ae) is similarly treated by defining, 


U = Y724 » Ve 15725 ‘ 


K, (5) = P(U- + v- s 9° ) = | i) Ey 6») dv du 


u2ny2 < 67 
where 
it we Cutz D 
Eo) = pF lg = pH @ for u + zy 20 
= 0 otherwise 
and 
“4 (vtz ) 
f (v) =f (wiz) =He Z 2 forv+z 20 
V Yo 2 2 
= 0 otherwise. 


The functional form of re will be the same as HL) K, (6) 


and dk, (5) will be defined over the same regions as H, (6) « 
Evaluating H, (5) » K, (5) and dK, (5) and combining terms yields, 
when Z 22, 

1 2 


: - eve 
12, doz ee 174 r 025 


z EK -\ - 
P . a | 2 (ee, “S224 ba 


oO 


“Ays-h, V 62-57 


=|4.Z.=[bZ “ib. 2,- Zz, -b_ 5 
ds)" [1 + 4e 11°22 -4e LY tee! 


u-y J 62-u2 en | 


“|b. Z,-ls,z “HB 
- diame tema | He ae du] 


0 


§ 
rX 
Pye 
@) 


“b.2,-b,z —w_6 -b.Z -b z 
Bs wl 2 2 an is aes 


[-8, i 
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= “hiz “A 2.-A 2 ¢° -h eed Wo omen 
oir | phn 


l 
tal [l-2e E sinh r6 +e i 4 


ds] 
=, ise 
6 2.2 
“Wz “.Z.-) Z ~ “Hh yup V 6 -u n-l 
[l-2e Me sinh W,6 +e Pl22 e 2 du] 
-§ 
“bz wh ZU “Wb Zy-bz 
[2 e it cosh ».6 - 2 e pie22 cosh w.6 +u 6 e 1 22 
1 1 1 2 
6 ee y g2-u2 
J ed du] dé 
asf g2- y2 


Lee) 


-X -X § 2 
oe | [ deel aha oat ne 979 ° “Aj s- = has n 


‘oi ds | 
Z 
- -Z 
it 
6 9,2 
“Zw 6 “lb Z2,°M Z “lb u- 8 ‘a | 
1 ia he u n 
[e 1 pe 1 + e 1 22 je l 2 du] 
“zy 
“pb Z -1b_6 “,Z.-» Z.- 6 “b.Z.-b Z 
[u¢ ee i = tee fe U6 ° l1l‘9o@2 








6 cb u-p, ¥ 5 “ 
l 2 
ae 
| du] dé 
d pas 
Zs § u 


and when z 2 Zs 


z -h_z,-A,2 -A,z_-A z -A 6 -Az-Az 
B® =n | ines. 4 oy age eee ae er ee 


Oo 


8 
P  aAys-A, Y82-s7 in 
J Ave 2 ds] 


“be -b_Z = 6 “[ 
i - 4e é 


“b.Z%_-[b Z 
fae an ae 2 I as 


6 2 
7B, Mote 67-u Pea! 
| Hoe 2 du] 


Oo 


“b.Z.~b,Z, 715.6 “) Zz —) Z 
oa L122 +g 1} 1 22 
1 1 
§ Zoe 
On ite §“~- 
Hobe |} =) , 
ta du] dé 
) [Ding 
z hi zee eee 
r | l 2 
hae J 4 - 2 hje eee ie sinh eee ds)" 
Zz 
ae ; 
6 en u l 


“BZ HZ, 2 n- 
a ee VsinhJ6*-u du] 


aa 


[2.6 


§ -b.U 
-b4Z,-,z 1 }.2. 2 
be vl hie @  coshvé"-u a) 6 


al ea - " 
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P (z) = 
1 


n 
ds] 


z 


fe] 
5. 8) ..6 =) geen Be cee 2_.2 
+n i en A eae ms 27 2”9 { 4 ‘ais ae 6°-s 
2 a 


ne 522 


6 
“Hh Zim Zz ro tH Np 
fe tit 4g, 1122] «© 1 2 


n-l 
du } 


l 


fe Aad as) “42,7, 2, bb § “Zz 
Met) Lee 22 1 ee Ind e a 2 


6 -Wu-H §2-y? 
; 2 


Se du} dé 
2 
"2, Pr. 


P = | | Pz) 242, ce dz,dz, + | | P (z) 72, (eee ae 
= Z 22 
1 2 2 1 


Representative values of Py were obtained for the hyper-cube 


distance function and are listed in Table 1. Evaluation was by a 
computer program utilizing FORTRAN 63 and a sub-routine based on 
Legendre-Gaussian quadrature. Appendix I lists the program as used. 
2 Red 2 


also for reduced computer time. The program is general and allows 


The case i d was selected for comparison reasons and 
selection of any parameters greater than or equal to zero. 
Time limitation prevented attainment of comparative results for 


the Euclidean distance function. 
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A =) =10 |rA. =A. = 10 JA. =A_ = 10 


I” @ 1 2 igglaens 
Sl a al i = 20 b, = a = 30 Hy = be = 40 
1 5488 5285 5032 
4 .4709 3989 ~3417 
8 4463 | 3612 .2973 
20 | ~4259 Re | - 2682 
TABLE 1 


Probability of Error P_, Nonparametric Discriminator, Bivariate 


Exponential Distribution. Distance function, hyper-cube, A. 
k = 1, rule of nearest neighbor. 
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4. CONCLUSIONS AND ACKNOWLEDGEMENTS 


The application of Hodges and Fix's work to the two population 
bivariate exponential case in Section 3 resulted in representative 
results tabulated in Table 1. Though the results are small in number, 
they allow comparison with that presented by Hager in [§]. 

The nonparametric discriminator error probability values in [§] 
indicate similarity to that listed in Table 1. The case when 
A =r >p # a corresponds to the case c > 1 in [S$]. In both [§] 


1 2 1 
and Table 1, probability of error a increases with increase in n. 


AK =i < 
Likewise 1 2 aa 


increasing n in both Table 1 and [§$]. Table 1 does not indicate the 


= H is analgous to c < l and By decreases with 


sensitivity of P, to change in parameter magnitudes nor the effect of 


1 
the change of a single parameter. 

The intent of this thesis was to develop the explicit forms of 
the error probabilities for the two distance functions and then eval- 
uate the probabilities to determine if there is any superiority of 
One distance function over another. This goal was only partially 
attained due to the programming complexity and extensive computer time 
required to evaluate the error probability. 

The following are recommended as areas Of further work: 

l. Examine sensitivity of 7 to change in parameter magnitudes 
and variation of a single parameter. 

2. Attempt to streamline program or approximation method to 
shorten computer time. 

3. Compute Pi for the Euclidean distance function to compare 


results with the hyper-cube. 
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4. Develop functional relationship of PF and Po when p = 2. 


5. Examine other distance functions. 


I wish to thank Professor J. R. Borsting for his enlightening 
guidance and assistance in preparing this thesis. I also wish to 
extend my gratitude to Mrs. Patricia Johnson for her programming 
assistance in evaluating the equations of Section 3 and to the Computer 


Facility for their help in processing the programs. 
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APPENDIX I 


t 
PROGRAMMTHRECUBE 
COMMON 81sB2:D1sD2sNsXNsINDs IMD 
1 READ 10000NsB1sB2:sD1 D2 
PRINT29N28B198B23s0D1sD2 
2 FORMAT (2X s2HN2®1592Xs3HB1=EL5F ce Bs 2X5 3HB2=E150893X> 3HO12®E15e8s2Xe3HD2 
1=E156¢8///) 
1000 FORMAT (15s4F10-0) 
XN=N 
IND=1 
CALL TRAP 1(A1) 
PRINT3sAL. 
3 Melty Fig =E15e8///) 
IND=2 
CALL TRAP 1 (A2) 
PRINT 4sA2 
4 FORMAT(5Xs3HA2Z=E15e8///) 
Pl = XN*(A1+A2) 
PRINT 20009N9B12829D1sD2sP1 
2000 FORMAT (2Xs2HN=1592X0 3HB1=E150892Xs 3HB2=E150892X9 BHDIZE15 08 9 2X 
1 3HD2=E154¢8s2Xe3HP1=E15e8///) 
GO TO l 
END 
SUBROUTINE TRAP1(AREA) 
COMMON Bl9B2¢° D1eD2eNsXNs INDs IMD 


ee 


NN=10 

XINC = e¢5 
XEND = Oc 
AREA = Oc 


10 XI = XEND 
XEND = XEND +XINC 
AREAX = GLQUAD (XI «XENDoNN) 
AREA = AREA+AREAX 
IF (AREAX~-1eE-06) 20920910 
20 RETURN 
END 
FUNCTION F(U) 
COMMON B1sB29D1]1sD29NsXNsINDs IMD 
NN=10 
XINC ca 
AREA = Oc 
XEND = Oc 
IF (ABSF (U)—-1-e¢E-06) 30930910 
10 XI=XEND 
XEND = XEND +XINC 
IF (XEND-U) 15915012 
12 XEND = U 
15 AREAX = GLQUAD2 (XI »sXENDoNNoVU) 
AREA =AREA + AREAX 
IF (ABSF (U-=XEND)—-1eE-06) 30930520 
20 IF CAREAX~-1-E-06) 30930910 
30 GO TO (40550)eIND 
40 8=B1 
GO TO 60 
50 B=B2 
60 F = B*¥EXPF(-B#U) #AREA 
RETURN 
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END 
FUNCTION G(Vo9VU) 
COMMON B12B29D1sD2sNsXNsoINDs IMD 


IMD=1 

CALL TRAP2 (OsVsUsVosAl1) 
IMD=2 

CALL TRAP2 (VsUsUsVsA2) 
IMD=3 


CALL TRAP 3( UsUsVsA3) 
GO TO (400%500)sIND | 
400 B=Bl1 ' 
GO TO 600 
500 B=B2 
600 G = B¥ EXPF(~B*¥V) *(A1+A2+A3) | 
RETURN 
END 
SUBROUTINE TRAP2(AsCsUsVsAREA) . 
COMMON 81982sD12D2 »NesXNoINDs IMD 
NN=10 | 
XUNG@=_,5 3 
AREA=0 6 
XEND=A 
IF (ABSF (C)-12E-06) 30930310 
10 XI=XEND 
XEND = XEND +XINC 
IF (XEND~C) 15915912 
12 XEND = C 
15 AREAX = GLQUAD 3 (XI »XENDoNNoUsV) 
AREA=AREA+AREAX 
IF (ABSF (C-=XEND)-12E-06) 30230220 
20 IF(AREAX~1¢e¢E~-06) 30930910 
30 RETURN 
END 
SUBROUTINE TRAP3(AsUsVsAREA) 
COMMON B19829D1sD29NeXNoINDo IMD 
NN=10 
XINC=.5 
AREA=06 
XEND=A 
10 XI=XEND 
XEND = XEND +XINC 
AREAX = GLQUAD 3 (XI sXENDsNNoUsV) 
AREA=AREA+AREAX 
IF (AREAX~12e¢E-06)20320910 
20 RETURN 
END 
FUNCTION HF (X9UsV) 
COMMON B1sB29D1sD2 »N»sXNsINDs IMD 
GO TO ( 1005500) »sIND 
100 21=U 
Z2=V 
GO TO (200,3003400)sIMD 
200 HF= (16e7~4e*EXPF(-B1*Z1-B2*Z2)%* SINH(XsB2)*SINH(XoB1] ) )#NE( Lembo # 
LEXPF( -D1*Z1-D2*Z2)* SINH(D29X)*SINH(D1 9X) ) *¥¥(N-1)*( 40% EXPF(-01% 
221-D2*Z2)*(DI*SINH(D2 9X) *COSH(D19X)+D2*SINH( D1 9X) *COSH( D2 9X) ) ) 
GO TO 900 ° 


40 





ANNAN 


300 
1 
2 
3 
4 


400 
1 
2 
3 


500 


600 
1 
2 


700 
1 
2 
3 
4 


800 
M 
2 
3 

900 


D 


D 


D 
D 


D 
D 


HF = (le -2e*EXPF(-B1*Z1)* SINH(Bl9X) +20 *EXPF(-B1*Z1-B2*Z2-B2*X ) 
%* SINH(B1l9X))*#N¥(1le— 20*EXPF(-D1¥Z1)*SINH(D19X) +20*EXPF(-D1#21 


=D2*Z2-D2*X) we SINHO(D1 9X) WA* (N=DI*FC 2¢*D1 ¥EXPE( -DIZ 1) * COSH 
(D19X) = 2¢*D1*¥EXPF(-D1*Z1-D2*Z2 -D2*X) * COSH(D19X) +2e*D2*EXPF 
(-D1*Z1-D2*#Z2-D2*X) *¥SINH(D15X)) 

GO TO 9060 

HF= (EXPF(-B1*Z1-Bl*X) +EXPF(=-B2*Z22-B2*X) -EXPF (-B1*Z1-B2*22-B1* 


X-B2*X )l) HENX (CEXPF(-D1¥*Z1-D1*¥X) +EXPF(-D2*Z2-D2*X) -EXPF(-D1#*Z1- 
D2*Z2 -QL*X-D2*X))**(N-1) *(D1#EXPF(-D1*Z1-D1*X) +D2* EXPF(-D2* 


Z22-D2*X)-(D1+D2) *EXPF(-D1¥*Z1-D2*Z2-D1*X-D2*X) ) 
GO TO 900 
21=V 
Z2=U 
GO TO (600+%700s800)sIMD 
HF= (10-4. *EXPF(-B1*Z1-B2*Z2)* SINH(X9B2)*SINHEX9B1) ) ¥¥N¥( 1 e—4e* 
EXPF( ~D1¥21-D2*Z2)* SINH(D29X)*SINH(D1 9X) ) #*#(N=-1)*(4e% EXPF(-D1* 
Z21-D2*Z2)*(D1¥*SINH(D2 9X) *COSH(N19X)+D2*SINH(D19X) *COSH(D29X))) 


GO TO 900 
HF=(1. -2e*EXPF (-B2*2Z2)*SINH(B29X) +20 *EXPF(-B1*Z1-B2*Z22-B1*X ) * 
SINH(B2 9X) )¥*¥*¥N¥(Le-20e% EXPF(-D2*Z2)* SINH(D29X)+ 2e* EXPF(-D1*2Z1-D 


2*Z2-D1¥*X) *SINH(D29X) )¥**(N-1) ¥(2¢*D2*EXPF(-D2*Z2)* COSH(D2 9X) 
—2e* D2%EXPF (=DIPOEZT=D2"Z 2 -DL*K) FCOSH(D2eX) +26¢*D) *EXPP US pier Z 
¥#Z2 -D1#¥X)* SINH(D29X)) 


GO TO 900 

HF=. (EXPPX=]B2 #2 1=BexX) HEXPF I= B2R7Z 2=B2* xX) 
X-B2*X))*¥*#¥N* (EXPF(-D1*Z1-D1*X) 
D2*Z2 -D1*¥X-D2*X) ) ¥*(N—-1) 
Z2-D2*X)-(D1+D2) 

RETURN 

END 

FUNCTION COSH(Y 9X) 

COSH=(E MRF ( YotX ) +E MRF (-Y*#X))/26 
RETURN 

END 

FUNCTION SINH(Y 0X) 
SINH=(EXPF (Y#X) -EXPF(-Y*¥X))/2.6 

RETURN 

END 

FUNCTION GLQUAD 
D1 UCSD GLQUAD 

GAUSSITAN-LEGENDRE QUADRATURE OF F FROM A TO Bs 


-EXPF (~B1*Z21-B82*2Z2-B1* 
+EXPF (-D2*Z2-D2*X) -EXPF(-D1*Z1- 
*(DL*¥EXPF(-D1*Z12=D1*X) +D2* EXPF(-D2* 
#EXPF (-D1*Z1-D2*Z22-D1*X=-D2*xX) ) 


(AsBoN) 


10920 OR 40 NODES. 


Pe YAGER 10/20/64 (RFFe KRYLOV PP3389341 AND SEC 72) 

COMMON /GLQDATA/X1(5) 9A1(5)9X2(10) sA2(10) 9X4( 20) 5A4( 20) 

DATA (X1= : 
097390652855 «8650633667 


DATA (Al= 


067940956835 643339539415 1488743390) 


DATA 


DATA 


DATA 


006667134435 
(X25 
09931285992 » 
©6360536807> 
(A2= 
01761400719 
01181945320, 
(X4= 


01494513491» 


S640 7192 740 
5108670020». 


00406014298 
1316886384, 


41 


2190863625, 


09172344283 > 
3737060887, 


00626720483, 
© 14209610935 


©2692667193+ 


8391169718» 
02277858511> 


00832767416» 
014917298655 


02955242247) 


© 7463319065» 
00765265211) 


©1019301198, 
01527533871) 


NAOAOAN 


\ 


own £& WN ke 


co ~ 


UU 090 


0 
D 
D 
D 


DATA 


099823770979 
°©9020988070> 
06719566846> 
© 3419940908 » 
(A4= 

©e0045212771>» 
©09334601953 4 
005743976915 
00728865824, 


29907262387» 
» 86595950325 
06125538897» 
© 2681521850» 


00104982845» 
003878216809 
06130624259 
00747231691 


91 12595500% 
8246122308» 
054946712515 
019269758075 


00164210584, 
094387090825 
©0648040135>% 
00761103619 » 


oF ITO Poemss 2s 
07783056514, 
048307580179 
© 1160840707> 


00222458492» 
00486958076» 
206791204585 
00770398182+ 


9328128083, 


07273182552, 
04137792044, 
00387724175) 


0279370070» 


00532278470, © 
00706116474, | 
00775059480) 


TO=(A+B)/2e § 
IF (N-10}19194 
DO 2 K=195 
Y=Y+A1L(K)¥*¥(F(TO=T1*¥X1(K) )+F(TO+TL¥X1(K))) 
GLQUAD=Y*T1l 

RETURN 

IF (N-20)59597 

DO 6 K=1910 

Y=YtA2(K) (FC TO-T1*¥X2(K) FF (TO+TLEX2(K) )) 
GOTO 3 i 
DO 8 K=1+920 


T1=(B-A)/20e S$ Y=O0 


2468 OURTEPPPEREN pret po 0 76 


YV=EY+tA4S(KIEC FEC TOKTIFXG (KD +FCTOFTLEXG(K) )) 


D 


D 


D 
D 


D 
D 


D 
D 
D 
D 


D 
D 
D 
D 


GOTO 3 : 
END 

FUNCTION GLQUAD2 
D1 UCSD GLQUAD 
GAUSSTAN=LEGENDRE QUADRATURE OF F FROM A TO Bs 


(AsBoNoVU) 


P. YAGER 10/20/64 (REF. KRYLOV PP3389341 AND SEC 7e2) 


COMMON /GLQDATA/X1(5)9A1(5)9X2(10) 9A2(10) 9X4(020) sA4(20) 
DATA (X1= 


DATA 


DATA 


DATA 


DATA 


DATA 


09739065285, 
(Al= 

00666713443, 
(X2= 

99312859929 
©6360536807>» 
(A2= 

001761400715 
01181945320» 
(X4= 

099823770975 
090209880705 
06719566846, 
© 34199409085 
(A4= 

000452127715 
00334601953» 
005743976919 
007288658249 


«8650633667 
014945134915 


09639719273» 
©5108670020% 


°©0406014298, 
1316886384, 


09907262387 
8659595032» 
061255388975 
©2681521850% 


00104982845, 
003878216809 
006130624259 
©0747231691>5 


06794095683 > 
21908636259 


9122344283» 
3737060887 > 


00626720483 » 
1420961093 


09772599500> 
82461223085 
054946712519 
019269758075 


00164210584, 
00438709082 » 
0648040135, 
20761103619» 


043339539415 
26926671935 


©8391169718>s 
022778585115 


00832767416» 
14917298655 


095791681925 
077830565149 
04830758017» 
01160840707 > 


00222458492 4 
00486958076» 
006791204589 
007703981825 


© 1488743390) 


10%20 OR 40 NODES. 


~. a 


02955242247) . 


© 74633190659 
©0765265211) 


©1019301198, 


1527533871) 


09328128083, 
0/273182552,5 
04137792044, 
00387724175) 


0e0279370070% 


00532278470>. 


e0706116474> 


20775059480) 


TO=(A+B)/26 S$ T1=(B-A)/20¢ S$ Y=O 


IF(N-10)191594 
1 DO 2 K=195 , 
2 Y=Y+tAl(K)*#(GCTO-T1L¥EX1(K) 9U)4+G( TO+T1*¥X1(K) VU) ) 
3 GLOQUAD2 =Y*TI1 

RETURN : 
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4 IF (N-20459597 
5 DO 6 K=1510 
6 Y=Y+A2(K) *(G(TO-T1¥X2(K) 9U)+G(TO+T1*X2(K) 9U) ) 
GOTO 3 : 
7 DO 8 K=1520 
8B Y=YtA4(K)*(GI(TO-T1¥X4(K) 9U)+G(TO+T1¥X4(K) 9U)) 
GOTO 3 
END 
FUNCTION+GLQUAD3 (AsBsNoUsV) 
C D1 UCSD GLQUAD 
C GAUSSIAN-LEGENDRE QUADRATURE OF F FROM A TO By 10920 OR 40 NODES. 
C 
(a P. YAGER 10/20/64 (REFe KRYLOV PP3385341 AND SEC 7e2) 
C 
COMMON /GLQDATA/X1(5) 9A1(5)9X2(10) 9A2(10) 9X4(20) 9A4(20) 
DATA (X1= 1 
D 097390652855 68650633667% 667940956835 043339539415 1488743390) 
DATA (Al= 
D 006667134439 014945134919 02190863625% 6269266171939 2955242247) 
DATA (X2= 
D 099312859925 696397192739 91223442835 083911697185 7463319065> 
D 06360536807 05108670020s 037370608875 e227785851lls 00765265211) 
DATA (A2= 
D 00176140071+ 200406014298s 06267204839 608327674165 21019301198, 
D 211819453205 413168863845 614209610935 1491729865» 1527533871) 
DATA (X4= 3 
D 09982377097s 699072623879 0697725995005 695791681925 9328128083, 
D 090209880709 68659595032 682461223085 67783056514 72731825523 
D 06719566846 661255388975 65494671251 048307580179 041377920445 
D 234199409085 626815218505 19269758079 11608407079 00387724175) 
DATA (A4= 
D 000452127715 001049828455 601642105845 002224584925 09279370070» 
D 203346019535 20387821680 204387090825 e0486958076s 005322784705 
D 0605743976915 606130624255 606480401355 2606791204585 0706116474, 
D 00728865824, 407472316915 200761103619s 07703981825 0775059480) 
TO=(A4+B)/2¢ $ T1=(B-A)/2¢ $ Y=0 
IF(N-10)19194 
1 DO 2 K=l1l95 
2 Y=Y+A1(K)*(HF (TO-T1*X1(K) s9UsV)+HFE (TO+T1EX1(K) 9UsV)) 
3 GLQUAD3 =Y*T1 
RETURN 
4 IF(N-20)59597 
5 DO 6 K=1910 
6 Y=Y+tA2(K)*CHF (TO-T1¥X2(K) 9U9V)+HF( TO+T1*X2(K) 9UsV)) 
| GOTO 3 
7 DO 8 K=1920 
8 Y=Y+tA4(K)*(HF(TO-T1¥XG(K) 9U9V) +HF(TO+T1#*X4G(K) 9UsV) ) 
GOTO 3 
END 
END 
FINIS 
“EXECUTE. 
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